1. Field of the Invention
The present invention relates generally to image data, and in particular, to a method, system, apparatus, computer program produce, and article of manufacture for combining two-dimensional (2D) images with depth images or point clouds for an accurate and robust extraction of junctions.
2. Description of the Related Art
Detecting junctions or edges in three-dimensional (3D) reality capture data is important for different applications. For example, extracting meaningful information such as the junctions of surfaces (e.g., walls, doors in a building, etc.) from a basic 3D point cloud can be used to re-create high-level computer aided design (CAD) descriptions from a point cloud. In another example, detecting junctions may be used to find specific characteristic points in several point clouds shot from different locations, and using the points to register the 3D point clouds together in space. Prior art applications fail to provide low level processing capabilities that detect edges/junctions from such data.
Different types of devices/sensors are used for different purposes—a first type of sensor may be used for color/intensity (e.g., a laser scanner that obtains point cloud data) while a second type of sensor may be used for depth (e.g., a Kinect™ sensor). Low-level processing modules (i.e., modules that examine small windows/parts of the data) are used to extract/determine information/data from color/intensity data and depth data. For example, low-level processing modules may be used to extract/determine edges/corners in images. Similarly, low-level processing modules may be used to extract/determine information in depth data.
While images that provide color/intensity data may be useful at showing where edges/transitions occur (e.g., one can see the edges in mages), such images are poor at perceiving depth (especially at edges). For example, a laser may cast a ray on a surface and examine what is returned to the laser device. Unfortunately, many discontinuities are returned wherein one part of the ray may hit a near surface while another part of the ray hits a far surface. If there is a sharp angle on the surface and both faces of the surface are visible, the different directions of the surface may impact the ray in different ways resulting in a returned signal that is unreliable or inaccurate. Similarly, depth based sensors capture depth with a sparse resolution or a resolution that is not as dense as that obtained with color/intensity based scanners (e.g., that acquire RGB [red, green, blue] pixel data).
In a CAD environment, accuracy is of the utmost important. For example, users may desire to model beam structures and capture edges of structures (e.g., doors, windows, and walls). Edge measurements provide the accurate position of the beam elements. However, most CAD users rely on laser scanners. While laser scanners may capture flat surfaces and planes well, edges are not easily determinable/captured. To obtain edges, most prior art systems intersect planes. However, the sampling of laser points will not find points on an edge but points close to an edge (e.g., on one side or the other) thereby requiring a user to examine neighboring points to locate an edge. Such an examination is expensive in terms of time and processing and is also inaccurate.
A light camera/scanner may be able to obtain additional information (e.g., higher resolution data) where a depth based scanner is lacking in quality. Pictures and photographs may be used to gather light for all of the surfaces (e.g., by capturing all light emitted and returned in a scanner/camera). With a light laser scanner, the sampling may be sparse with points that are very disjointed. However, there is no capability in the prior art that combines depth data from a light laser scanner with that of a color/intensity based scanner.
What is needed is the capability to combine data from a laser scanner with depth based data in order to accurately determine junctions and edges.